Improving UCT planning via approximate homomorphisms
نویسندگان
چکیده
In this paper we show how abstractions can help UCT’s performance. Ideal abstractions are homomorphisms because they preserve optimal policies, but they rarely exist, and are computationally hard to find even when they do. We show how a combination of (i) finding local abstractions in the layered-DAG MDP induced by a set of UCT trajectories (rather than finding abstractions in the global MDP), and (ii) accepting approximate homomorphisms, leads to greater prevalence of good abstractions and makes them computationally easier to find. We propose an algorithm for finding abstractions in UCT planning and derive a lower bound on its performance. We show empirically that it improves performance on illustrative tasks, and on the game of Othello.
منابع مشابه
Discrete Asymptotic
We obtain six-terms exact sequences for E-theory and KK-theory which involve discrete asymptotic homomorphisms, and generalize the extension of groups from the UCT theorem.
متن کاملImproving Exploration in UCT Using Local Manifolds
Monte-Carlo planning has been proven successful in many sequential decision-making settings, but it suffers from poor exploration when the rewards are sparse. In this paper, we improve exploration in UCT by generalizing across similar states using a given distance metric. We show that this algorithm, like UCT, converges asymptotically to the optimal action. When the state space does not have a ...
متن کاملQuasidiagonal Extensions and Sequentially Trivial Asymptotic Homomorphisms
to be quasidiagonal when B⊗K contains an approximate unit of projections which is quasi-central in E, cf. [Sa]. He identified, under certain conditions, the subgroup of KK(A,B) which the quasidiagonal extensions correspond to under Kasparov’s isomorphism Ext(A,B) ' KK(A,B). C. Schochet has removed some of Salinas’ conditions in [S], the result being that when A is a unital C-algebra in the boot...
متن کاملApproximate unitary equivalence and the topology of Ext(A,B)
Let A, B be unital C*-algebras and assume that A is separable and quasidiagonal relative to B. Let φ,ψ : A → B be unital ∗-homomorphisms. If A is nuclear and satisfies the UCT, we prove that φ is approximately stably unitarily equivalent to ψ if and only if φ∗ = ψ∗ : K∗(A,Z/n) → K∗(B,Z/n) for all n ≥ 0. We give a new proof of a result of [DE2] which states that if A is separable and quasidiagon...
متن کاملMorphisms of Simple Tracially Af Algebras
Let A, B be separable simple unital tracially AF C*-algebras. Assuming that A is exact and satisfies the Universal Coefficient Theorem (UCT) in KK-theory, we prove the existence, and uniqueness modulo approximately inner automorphisms, of nuclear ∗-homomorphisms from A to B with prescribed K-theory data. This implies the AF-embeddability of separable exact residually finite dimensional C*-algeb...
متن کامل